System and method for acquiring images at maximum acquisition rate while asynchronously sequencing microscope devices

ABSTRACT

The invention provides an automated microscope system, a computer program product which may be programmed into the automated microscope system and a method for acquiring images at substantially the maximum acquisition rate of a camera while and as devices external to the camera, which vary various parameters of the acquired images, operate asynchronously with the camera so as to allow the acquired images to be displayed as a sequence that shows continuous variation in the acquisition parameters.

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation of U.S. patent applicationSer. No. 09/638,548, filed Aug. 14, 2000, which issued on Apr. 20, 2004,as U.S. Pat. No. 6,724,419. This application claims the benefit of U.S.Provisional Application No. 60/148,819, filed Aug. 13, 1999. The entirecontents of all of the aforementioned documents are incorporated hereinby reference.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] This invention relates to the field of acquiring images using anautomated optical microscope system that includes a camera andparticularly, to a method and system for acquiring images as devicesexternal to the camera, which alter various parameters of the acquiredimages, operate asynchronously with the camera so as to allow suchimages to be displayed as a sequence that shows continuous variation inthe acquisition parameters.

[0004] 2. Discussion of the Prior Art

[0005] Performing research on living cells demands the use of anautomated microscope system controlled by application software. Withparticular reference to the fluorescence microscopy of cells andtissues, various methods for imaging fluorescently-stained cells in amicroscope and for extracting information about the spatial and temporalchanges occurring in these cells are well known in the art. An articleby Taylor, et al. in AMERICAN SCIENTIST 80 (1992), p. 322-335 describesmany of these methods and their applications. Such methods have beenparticularly designed for the preparation of fluorescent reportermolecules so as to yield, after processing and analysis, spatial andtemporal resolution imaging measurements of the distribution, the amountand the biochemical environment of these molecules in living cells.

[0006] As regards automating microscope systems, it is well known in theart that automated microscope systems may arrange any one of a widevariety of cameras and an array of hardware components into variousinstrumentation configurations, depending on the specialized researchtask at hand. A standard reference, especially useful for an expositionof automated optical microscopy hardware, hardware systems and systemintegration, is VIDEO MICROSCOPY, 2d Ed., 1997 by Inoué and Spring,which is incorporated herein by reference, especially Chapter 5. Moregenerally descriptive of automated image acquisition andthree-dimensional image visualization is John Russ's, THE IMAGEPROCESSING HANDBOOK, 3d Ed., 1999, pp: 1-86, 617-688 and referencestherein.

[0007] Also well known is that an application software package maysupplement and overlay a particular instrumentation configuration bycontrolling and specifying the sequence, way, and functionalities ofimage acquiring and processing that the instrumentation system performs.The acquiring and processing operations that the software package iscalled upon to do depends, again, on the specialized research task athand. The chapter titled “A High-Resolution Multimode Digital MicroscopeSystem” by Salmon et al. in METHODS IN CELL BIOLOGY, VOL. 56, ed. bySluder & Wolf, 1998, pp:185-215 discusses the design of a hardwaresystem, including the microscope, camera, and Z-axis focus device of anautomated optical microscope as well as application software forautomating the microscope and controlling the camera.

[0008] Existing application software for automating a microscope systemcan direct and control a host of operations, including:

[0009] image acquisition from Recommended Standards (“RS”)-170 videodevices, charge-coupled devices, NTSC and PAL video sources;

[0010] setting exposure time, gain, analog to digital conversion time,and bits per pixel for camera settings at each emission and/orexcitation wavelength;

[0011] driving the digitizing of acquired images from an analog todigital converter;

[0012] storing acquired images in a variety of formats, such as TIFF,BMP, and other standard file formats;

[0013] driving microscope illumination;

[0014] providing capability of creating macros from a user-specifiedsequence of program commands, which are saved and recorded and able tobe played back at a single click; performing certain processes on agroup of related images, called a stack, such as aligning images withinthe stack, rendering a 3-dimensional reconstruction, saving the stack toa disk, enhancing the images, deblurring the images, performingarithmetic operations; and analyzing image parameters, such as ratioimaging the concentration of ions and graphing changes in intensity andin ratios of ion concentration over time.

[0015] An example of widely-used, prior application software forautomating a microscope system is the Meta Imaging Series™ availablefrom Universal Imaging Corporation, West Chester, Pa., which is aconstellation of related application programs, each having a differentpurpose. For example, a user wanting a general, multipurpose imageacquisition and processing application would employ the MetaMorph™application program; while a user needing to perform ratiometricanalysis of intracellular ion measurements would employ MetaFluor™.

[0016] Notwithstanding the above list of operations that priorapplication software can direct an automated microscope system to do,prior application software has not heretofore enabled an automatedmicroscope system to acquire a group of images while and as acquisitionparameters, such as the focus position and the emission and/orexcitation wavelength, vary so that the acquired group of images can beplayed back as a sequence that shows continuous change in thoseparameters. That is, prior application software has not been capable ofdirecting external devices that control image acquisition parameters tooperate asychronously with the camera in order to acquire a group ofimages that may displayed as sequence showing continuous change insystem parameters.

[0017] Acquiring a group of images asynchronously as a biological eventis occurring so that the images can be played back as a sequencedisplaying continuous change in certain parameters of the microscopesystem has enormous importance in research with living cells. Theimportance of the present invention to cell research may be analogizedto the importance of time-lapse photography to the study of macroscopicliving systems. However, to be clear, the present invention is notmerely a method akin to time-lapse photography of images acquired ofliving structures and processes at the cellular level. Using priorapplication software for processing images of cellular structures andmechanisms, a researcher is unable to observe a continuous stream ofimages that show uninterrupted change in system parameters other thantime. The present invention allows a researcher to vary parameters, suchas the position of the lens objective and the emission and/or excitationwavelength, during image acquisition so that on playback the acquiredset of images may display this variability as continuous change.Specific examples of the kind of research that benefits from using thecurrent invention include observing the movement of adhered proteins onthe cell surface during live T-cell to B-cell cell-(immune cells)interactions and verifying a software model of diffusion of chemicalsintroduced into cells.

[0018] The following technical problem has remained unresolved by priorapplication software for automating an optical microscope system:namely, how to acquire images using a camera in an optical microscopesystem, operating at close to its maximum rate of acquisition, at thesame time that external devices to the camera are continuously changingthe settings of various parameters of image acquisition. The presentinvention solves this technical problem by providing a computerizedmethod whereby the camera and the external devices in an automatedoptical microscope system are instructed to operate asychronously, thatis, independently of each other, during image acquisition, therebyenabling the camera to acquire images that may be displayed as asequence showing continuous change in image acquisition parameters.

SUMMARY OF THE INVENTION

[0019] The present invention provides a method, a computer readablemedium and an automated optical microscope system for acquiring imagesat substantially the maximum acquisition rate of a camera while and asdevices external to the camera change acquisition parameters. The stackof images so acquired may be displayed as a sequence of images showingcontinuous change in the image acquisition parameters. Because thepresent invention directs external devices to change image acquisitionparameters while and as a camera is acquiring each frame in a set offrame images, instead of directing the devices to wait until the camerahas finished acquiring that frame, the camera and the external devicesoperate asynchronously. In different terms, the present inventiondirects external devices to operate to change the image acquisitionparameter they control, for example, the focus position of themicroscope objective lens, the emission and/or excitation wavelength orthe position of the microscope stage, while and as a camera is acquiringan image.

[0020] The method of the present invention comprises the followingsteps:

[0021] a) configuring an image-acquiring system comprising an automatedmicroscope, a camera, devices external to the camera for altering theimage acquisition parameters of focus plane, excitation wavelengthand/or emission wavelength and a computer, whereby the external devicesare directed to acquire images asychronously with the camera;

[0022] b) acquiring images at a rate substantially close to the maximumimage acquisition rate of the camera and storing the acquired images asdigitized data;

[0023] c) during the acquiring and storing of images, operating at leastone said external device whereby at least one image acquisitionparameter is altered;

[0024] The method of the present invention may be used under a widevariety of microscopy modes, including brightfield, fluorescence,darkfield, phase contrast, interference, and differential interferencecontrast (DIC). One of the embodiments of the method of the presentinvention is as a set of instructions resident in an informationhandling system. Another embodiment of the method of the presentinvention is as a set of instructions resident in a computer readablemedium.

[0025] It is a feature of the present invention that a group of images,called a stack, may be acquired as the focus position changes are madeso that the group of images so acquired may be displayed as a sequenceshowing continuous change of the Z-position. It is a further feature ofthe present invention that a stack of images may be acquired as changesto the emission and/or excitation wavelength are made so that a group ofimages may be displayed as a sequence showing continuous change of theemission and/or excitation wavelength. Further, it is a feature of thepresent invention that a stack of images may be acquired as changes tovarious acquisition parameters, such as the focus position and theemission and/or excitation wavelength are made in concert so that agroup of images displayed as a sequence show continuous change in thevarious acquisition parameters selected. A feature of another embodimentof the present invention is that at regular time intervals, a stack ofimages may be acquired while and as various image parameters aresimultaneously changed so that the stacks of images may be displayed asa sequence that shows continuous change over time and continuous changein acquisition parameters.

[0026] An advantage of the present invention is that a stack of imagesallows a three-dimensional rendering of the relevant cellular mechanism,process, and/or structure during a biological event, such as theintroduction of a chemical into a cell or cell division. A furtheradvantage is that multiple stacks of images allow a three-dimensionalrendering of a biological event of interest as image acquisitionparameters change and over a selected time period.

BRIEF DESCRIPTION OF THE DRAWINGS

[0027]FIG. 1 is a block diagram of an automated optical microscopesystem that provides the operating environment for an exemplaryembodiment of the present invention.

[0028]FIG. 2 shows a graphical representation of an exemplary hardwareconnection arrangement between an exemplary embodiment of an informationhandling subsystem and a microscope subsystem.

[0029]FIG. 3 shows a schematic diagram of an exemplary microscopesubsystem used to practice a method of the present invention.

[0030]FIG. 4 shows a graphical representation of an exemplary microscopeused to practice a method of the present invention.

[0031]FIG. 5 is a flow chart diagram depicting an exemplary embodimentof the overall method of the present invention.

[0032]FIG. 6A shows an exemplary user interface dialog for installing acamera into a system of the present invention.

[0033]FIG. 6B shows an exemplary user interface dialog for configuringparameters relating to the installed camera in FIG. 6A.

[0034]FIG. 6C shows an exemplary user interface dialog for installinginto a system of the present invention a device external to themicroscope that varies image acquisition parameters.

[0035]FIG. 6D shows an exemplary user interface dialog for configuringan installed objective lens positioner into a system of the presentinvention.

[0036]FIG. 7A shows the Acquire user interface dialog of the StreamAcquisition embodiment of a method of the present invention.

[0037]FIG. 7B shows the Focus user interface dialog of the StreamAcquisition embodiment of the present invention by which a user mayinput the starting and final focus position of an installed objectivelens positioner.

[0038]FIG. 7C shows the Wavelength user interface dialog of the StreamAcquisition embodiment of the present invention by which a user mayinput the wavelengths used to illuminate the specimen during imageacquisition.

[0039]FIG. 7D shows the Camera Parameters user interface dialog of theStream Acquisition embodiment of the present invention.

[0040]FIG. 8A shows the Main user interface dialog of theMultiDimensional Acquisition embodiment of the present invention.

[0041]FIG. 8B shows the Timelapse user interface dialog of theMultiDimensional Acquisition embodiment of the present invention bywhich a user may input the number of times a method of the presentinvention will acquire images in any one acquisition event.

[0042]FIG. 8C shows the Z Series user interface dialog of theMultiDimensional Acquisition embodiment of the present invention bywhich a user may input values for varying the Z-position whileperforming a method of the present invention.

[0043] FIGS. 8D-F show the Wavelengths user interface dialog of theMultiDimensional Acquisition embodiment of the present invention bywhich a user may input values for varying the wavelength whileperforming a method of the present invention.

[0044]FIG. 8G shows the Stream user interface dialog of theMultiDimensional Acquisition embodiment of the present invention.

[0045]FIG. 9 is a flow chart diagram depicting an exemplary embodimentof the steps of the Acquire routine of the present invention fordirecting a camera operating at or near its maximum rate to acquireimages while and as external devices operate asychronously with thecamera.

[0046]FIG. 10 is a flow chart diagram depicting an exemplary embodimentof the steps of a method of the present invention for directing externaldevices to vary image acquisition parameters while and as these devicesoperate asychronously with the camera.

[0047]FIG. 11 shows the Review MultiDimensional Data user interfacedialog of an exemplary image processing software application by which auser can playback a sequence of images created by a method of thepresent invention as a continuous stream or movie.

DETAILED DESCRIPTION

[0048] The present invention provides a method, a computer readablearticle of manufacture and an automated optical microscope system foracquiring images at substantially the maximum acquisition rate of acamera while and as external devices to the camera continuously changeimage acquisition parameters thereby allowing such images to bedisplayed as a sequence showing continuous variation in thoseparameters.

[0049] Definitions

[0050] Stack. As used herein, stack refers to the total set of imagesacquired by a method of the present invention during one acquisitionevent that may admit of image processing.

[0051] Focus Plane. As used herein, a focus plane is that verticalposition at which the object of interest is being observed and broughtinto visual focus.

[0052] Z-position. As used herein, Z-position refers to the focus plane,which is that vertical position at which the object of interest is beingobserved and brought into visual focus. Z-position is an imageacquisition parameter that may be varied by moving the objective lenspositioner or by moving the stage mover of an optical microscope.

[0053] Image Acquisition Parameter. As used herein, image acquisitionparameter includes the Z-position and the illumination wavelengths,which include either or both the excitation wavelength and the emissionwavelength.

[0054] Piezofocuser. As used herein, a piezofocuser is a short-hand termfor a piezoelectric objective lens positioner that operates to changethe focus position in speeds that range from 1 to 2 milliseconds.

[0055] Asynchronous operation. As used herein, asynchronous operationmeans the simultaneous operation of a camera in acquiring images whileand as external devices, such as an objective lens positioner, a stagemover and/or a wavelength changer, vary the specific image acquisitionparameter each controls.

[0056] Inter-Frame Time. As used herein, inter-frame time is the timebetween the end of exposing one frame and the start of exposing the nextframe. For a raster-scanned video camera, this is the time between theraster scan of the last digitized scan line of one frame and the rasterscan of the first digitized scan line of the next frame. For afull-frame CCD camera, this is the read out time of the full frame. Fora frame-transfer CCD camera, this is the frame transfer shift time.

[0057] General Organization of the System

[0058] FIGS. 1 to 4 and the following discussion are intended todescribe an exemplary optical microscope system of the presentinvention. The system of the present invention comprises an automatedoptical microscope system comprising an information handling subsystemfor executing a method of the present invention. The method comprisesacquiring images at substantially the maximum acquisition rate of acamera while and as external devices to the system microscope arechanging image acquisition parameters.

[0059] Those skilled in the art will appreciate that the invention maybe practiced with a variety of information handling configurations,including a personal computer system, microprocessors, hand-helddevices, multiprocessor systems, minicomputers, mainframe computers, andthe like. In addition, those skilled in the art will appreciate that theinvention may be practiced in distributed computing environments wheretasks are performed by local or remote processing devices that arelinked through a communications network or by linking to a serverconnected to the Internet through which the software method may beaccessed and used.

[0060] Moreover, those skilled in the art will appreciate that specificresearch goals and needs will dictate the configuration and specificapparatus used in an automated optical microscope system, especially asthese goals relate to processing images of living cells duringbiological events. The elements of an image processing system thatutilizes the program module of the present invention comprise amicroscope, a camera, means for processing the images according to theinstructions of the program module of the present invention, and atleast one means external to the microscope for varying at least oneimage acquisition parameter.

[0061]FIG. 1 shows a block diagram of an exemplary system of the presentinvention, having an optical microscope subsystem 100 and aninformation-handling subsystem 200. In this diagram, the microscopesubsystem 100 has been depicted as to its most generic parts and shouldnot be construed as limiting.

[0062] The exemplary microscope subsystem 100 comprises an opticalmicroscope 160, shown here for illustrative purposes as an uprightmicroscope. Devices external to the microscope 160 include anepi-illumination assembly 104, an emission wavelength changer 120 and acamera 130. The microscope 160 comprises a transmitted lightillumination arm 102, which comprises elements 106 through 110,including a transmitted light source 106—typically a halogen lamp, atransmitted light shutter 108, and a condenser 110. The microscope 160also comprises an ocular 122, a stage 126 and an objective lens 124.

[0063] The epi-illumination assembly 104 comprises elements 112 through116, including an epi-illumination light source 112, which forillustrative purposes comprises a fluorescence lamp 112, a shutter 114,and an excitation wavelength changer 116. Camera 130 is connected toboth the microscope subsystem 100 and the information handling system200.

[0064] The information handling subsystem 200 as shown in FIG. 1comprises an exemplary computer 211, illustrated here as a personalcomputer 211, comprising a central processing unit 213, a hard drive215, a CD-ROM 217, a floppy drive 219, a graphics adapter 221, anethernet network card 223, a PCI card 225 to interface computer 211 tocamera 130 and a Digital to Analog converter [DAC] card 227 to control apiezoelectric objective lens positioner (340 in FIG. 3) and, if includedin microscope subsystem 100, a monochromator fluorescence illuminator(not shown) that serves the same purpose as an excitation wavelengthchanger (116 in FIG. 1, 316 in FIG. 3). Peripherals in informationhandling system 200 comprise a graphics display monitor 231 and a videomonitor 241. Application software 245 allows control of the componentsof microscope subsystem 100, specifically controlling the illumination,focus and the camera as well as directing the image acquisition andprocessing. The method of the present invention may be operated as aprogram module of application software 245.

[0065]FIG. 2 shows a graphical representation of an exemplary hardwareconnection arrangement between an exemplary embodiment of informationhandling subsystem 200 and optical microscope subsystem 100. FIG. 2shows an embodiment of a computer in the form of a personal computer211. The back side 251 of computer 211 shows various serial and parallelports, 251, 261, 271, 281 and 291, and physical connections 253, 263,273, 283, and 293 by which computer 211 communicates with and controlsvarious external devices of optical microscope subsystem 100. The portsand connections shown include a port 251 from which a connection 253attaches to a camera (130 in FIG. 1, 330 in FIG. 3); a port 261 fromwhich a connection 263 attaches to the emission and/or excitationwavelengths changers (116 and 120 in FIG. 1, 316 and 320 in FIG. 3); aport 271 from which a connection 273 attaches to external shutters (114in FIG. 1, 314 in FIG. 3); a port 281 from which a connection 283attaches to a piezoelectric objective positioner (340 in FIG. 3); andport 291 from which a connection 293 attaches to a stage mover (336 inFIG. 3).

[0066]FIG. 3 shows a schematic diagram of another embodiment of anoptical microscope subsystem 300, used to practice the method of thepresent invention. Although the details of FIG. 3 parallel those of theblock diagram of microscope subsystem 100 shown in FIG. 1, FIG. 3provides a more graphical representation of the relationship between amicroscope and a camera used in a system of the present invention.

[0067] The microscope subsystem 300 in FIG. 3 comprises a microscope360, which for illustrative purposes, is depicted as an invertedmicroscope, as well as devices external to the microscope 360 whichinclude an epi-illumination assembly 304, an emission wavelength changer320 and a camera 330. Microscope 360 comprises a transmitted lightillumination arm 302, which comprises, a light source 306, exemplifiedas a halogen lamp 306, and a shutter 308. The epi-illumination assembly304 comprises an epi-illumination light source 312, here exemplified asa fluorescence lamp 312, an epi-illumination shutter 314, and anexcitation wavelength changer 316.

[0068] A specimen 332 to be observed is placed on microscope stage 326,which may be moved vertically up or down by stage mover 336, depicted inFIG. 3 as a mechanized stepper motor 336. A piezoelectric objective lenspositioner 340 encases objective lens 324 and moves lens 324 verticallyup or down in order to bring the specimen 332 in or out of visual focus.A revolving nosepiece 342 may be equipped with more than one objectivelens 324.

[0069] Camera 330 is connected to microscope subsystem 300 via cameraport 350. An emission wavelength changer 320 is placed betweenmicroscope 360 and camera 330.

[0070]FIG. 4 shows a graphical representation of an embodiment of amicroscope 460 used in a system of the present invention. Thetransmitted light illumination arm 402 houses a transmitted light 406and comprises a condenser 410. A researcher may manually bring aspecimen 432 placed on stage 426 into focus by looking through theocular 422 and operating a stage mover 436, depicted in FIG. 4 ascoarse/fine focusing knob 436. If desired, a stepper motor (not shown)may be attached to the focusing knob 436 to mechanize stage movement.

[0071] An epi-illumination assembly (not shown) that would comprise anepi-illumination light source, a shutter and an excitation wavelengthchanger (all not shown) could be attached to microscope 460 behind therevolving nosepiece 442 and objective lens 424. A piezoelectricobjective lens positioner (not shown) may be fitted over objective lens424. An emission wavelength changer (not shown) may be connected to thecamera port 450, which connects the microscope 460 to a camera (notshown).

[0072] How the Automated Microscope System Works

[0073] With continuing reference to FIG. 3, the automated microscopesystem 300 is directed by a method of the present invention to acquireimages as viewed through an optical microscope 360 and to store thoseimages as digitized data at or near the maximum acquisition rate of theacquiring camera 330. At the same time, the method is also directingexternal devices 340, 336, 316 and 320 to change a parameter related toimage acquisition, that is, the focus plane of the specimen and/orexcitation and/or emission wavelengths. A focus plane is that verticalposition at which the object of interest is being observed and broughtinto visual focus. A focus plane may be altered by changing the verticalposition of either objective lens 324 (by using objective lenspositioner 340) or stage 326 (by using stage mover 336).

[0074] As used herein, the term Z-position will refer to the focusplane, which may be changed by either an objective lens positioner 340or a stage mover 336.

[0075] Thus, system 300 of the present invention functions to acquireimages by having a programmed set of instructions to perform the methodof the present invention by directing camera 300 and external devices340, 336, 316 and 320 to operate asychronously during image acquisitionwhich is done at or near the maximum rate of camera 300. As pointed outabove, external devices 340, 336, 316 and 320 include objective lenspositioner 340 that moves objective lens 324 vertically up or down, astage mover 336 that moves microscope stage 326 vertically up or downand wavelength changers 316 and 320, which change the wavelength oflight used to excite the specimen and/or filters the light emitted fromspecimen 332.

[0076] To appreciate how the present invention works, it is important toremember that observations of living cellular material must occur atrates that correspond to the rates at which the observed processes areoccurring. Typically, camera 330 must acquire images of biologicalevents at the cellular level in the real time of milliseconds.

[0077] The Interaction Between a Camera, an Automated Microscope andExternal Devices

[0078] In observing events in living cells and tissues, microscopesubsystem 300 is set up, that is, configured, so that camera 330acquires images as essentially 2-D “snapshots” at different depths ofthe three-dimensional living structure at issue. Changing the verticalposition of objective lens 324 in effect changes the depth of the focusat which the 2-D “snapshot” is taken within the cell. Changing the focusdepth literally changes the perceptibility of different cellularstructures and processes that lie at different depths within the cell.Every time objective lens 324 changes position, a different focus isachieved, which brings a different visual perspective of internal cellstructure. Rapidly acquiring “snapshots” from a camera as the verticalposition of the focus changes results in a set of “photographs” that canbe displayed as a sequence or movie of images, which can on playbackallow a researcher, for example, to look from top to bottom (or viceversa) through a cell. A set of images acquired by system 300 for aparticular number of vertical positions (and/or selected wavelengths asdiscussed below) is a stack.

[0079] For example, if a researcher wishes to observe an event occurringat or on the membrane of the outer cell wall, typically a researcherwill first eye-focus microscope 300 by manually directing a mechanism336 to move stage 326 to a vertical position so that the outer cellmembrane comes into visual focus. Then, while camera 330 acquires imagesof the cell membrane, a researcher can opt to vary the focus position bymoving objective lens 324. In a system of the present invention, asoftware method of the present invention will direct a mechanized devicecalled a piezoelectric objective positioner 340, hereinafter called apiezofocuser, to vertically move objective lens 324 to focus ondifferent depths within the cell membrane. Each position to whichobjective lens 324 moves is termed in the art a Z-position. Sincedistances between Z-positions for cellular research are in terms ofnanometers, using a piezoelectric mechanism to move the objective lensallows the precise motion and positioning necessary without machineslippage and occurs typically at a rate of 2 milliseconds (ms). This ismost often sufficiently rapid to allow camera 330 to acquire a stack ofimages at different focal depths so that the stack displays on playbacka rapidly occurring biological event at the cell membrane.

[0080] In addition to varying the Z-positions of objective lens 324,subsystem 300 may also be configured to vary the wavelengths of lightused to illuminate specimen 332 during the acquisition of a stack ofimages. Cellular structures, processes and events of living cellularmaterial are often observed using fluorescence microscopy by introducingfluorescent chemical or protein probes. Further, the ability to solveproblems using fluorescence microscopy is enhanced by the systematicvarying of the wavelength of light used to excite a fluorescentlystained specimen or the filtering of the light that is emitted from thespecimen.

[0081] Different fluorophores, fluorescent chemical or proteinmolecules, absorb and become excited by electromagnetic radiation atspecific wavelengths of light and emit fluorescence at specificwavelengths of light. Moreover, cellular structures stained withdifferent fluorescent dyes are excited by and emit fluorescence withdifferent wavelength ranges of light. By restricting or filtering thewavelength of excitation light shone on a fluorescently stained specimenor the wavelength of emission light captured by camera 330, a researchercan observe different cellular structures or cellular events boththrough time and at different depths within a cell or tissue. Inaddition, by so restricting or filtering the fluorescence excitationand/or emission wavelengths, a researcher can highlight selected cellstructures or measure conditions within cellular structures.

[0082] For example, by staining a cellular organelle situated near theoutside cell membrane with fluorescent dye A and staining thechromosomes situated in the cell nucleus with fluorescent dye B, aresearcher can acquire a stack of images that focus on differentcellular events involving both organelles. This is so because of thenature of fluorescent dyes. Say, for example, that fluorescent dye A isexcited at wavelength A and fluorescent dye B excited at wavelength B.By switching back and forth between excitation wavelength A and B whilecamera 330 is taking 2-D “snapshots”, a researcher can create a stack ofimages which on playback can be processed to display structures that areexcited at only wavelength B or only wavelength A or at both wavelengthsand so can tease out from a composite cellular event the changes withinonly one element of interest. This concept is analogous to a TV-watcherusing a remote control to rapidly switch back and forth between channelsas a camera is taking pictures of the TV screen after each channelswitch.

[0083] In changing the excitation or emission wavelength, somewavelength changers can operate at a rate of 1.2 ms. For most cameras,this is sufficiently rapid to let camera 330 acquire a stack of imagesat different wavelengths so that the stack displays on playback arapidly occurring biological event involving more than one cellularelement or a biological process.

[0084] A system of the present invention may be set up to vary both theZ-position and the wavelength while camera 330 is acquiring images. Whena researcher opts to change both the Z-position and the wavelength, themethod of the present invention will first direct piezofocuser 340 tomove objective lens 324 to a new Z-position, and then direct wavelengthchanger 316 and/or 320 to switch to a new wavelength. Only after camera330 has finished acquiring images at each of the selected wavelengthswill the method then direct piezofocuser 340 to move to a newZ-position. In theory, the method allows a researcher to select anynumber of wavelengths to switch between. An embodiment of the presentinvention allows a researcher to select four different wavelengths.

[0085] As mentioned above, piezofocuser 340 has the capability to changethe Z-position of objective lens 324 in 2 ms and because wavelengthchanger 316 and/or 320 has the capability to change a wavelength in 1.2ms, the system-limiting component in the subsystem 300 is most oftencamera 330. Ultimately, the rate at which camera 330 acquires imagesdetermines whether there is sufficient time for external devices 340,336, 316 or 320 to change their respective parameters before the nextimage is acquired.

[0086] Operating Principles of the Method/System

[0087] At its essence, then, the method of the present invention relieson the difference between the acquisition rate of camera 330 and theoperation rate of piezofocuser 340, stage mover 336, wavelength changers316 and/or 320 in order to acquire a stack of images that can display onplayback continuous change in the selected parameters. A fundamentaloperating principle of the present invention is that when camera 330acquires images more slowly or almost as quickly as the external devicesoperate, then piezofocuser 340 and wavelength changers 316 and/or 320can change the Z-position and wavelength respectively before the camerastarts acquiring the next image. Put differently, camera 330 only has tobe marginally slower than the external devices 340, 336, 316 and/or 320in order for the Z-position and the wavelength to be varied quicklyenough so that the next acquired image will capture and be able todisplay these changed values.

[0088] A second fundamental operating principle of a method of thepresent invention is that image acquisition occurs while and aspiezofocuser 340, stage mover 336, wavelength changers 316 and/or 320are operating asynchronously with camera 330. Asynchronous operationmeans piezofocuser 340, stage mover 336, wavelength changers 316 and/or320 do not wait for a signal that the read out of the exposed image hasfinished before changing the Z-position and/or wavelength. Asynchronousoperation means that during camera readout, the external devices are areeither switching to the next desired wavelength and/or moving the focusplane to the next desired Z-position. In other words, camera 330 andexternal devices 340, 336, 316 and 320 are operating more or lesssimultaneously, and not sequentially. To the contrary, in a sequential,that is, synchronous, operation external devices 340, 336, 316 and/or320 would in fact wait for camera 330 to complete acquisition and readout before receiving a signal to change parameters. Synchronousoperation would therefore slow down the overall acquisition rate of thesubsystem 300, thereby disallowing it to acquire images at or near realthe maximum acquisition rate of camera 330, which is a primary object ofthe present invention.

[0089] Background to Cameras and Image Acquisition Rates

[0090] To provide a context and background for these essential operatingprinciples of the present invention, the following paragraphs brieflydiscuss image acquisition by a camera. Image acquisition by a camera isfully and well-documented in Inoué and Spring, Video Microscopy, 2d.Edition, 1997, incorporated herein by reference, especially Chapter 5.Suffice it to say here that a camera converts an optical image exposedon its photosensitive surface into a sequence of electrical signals thatmake up the video signal. For cameras using the North American (NTSC)format, the optical image is sampled, that is, read out-from top tobottom and from left to right—as if it were a rectangular page thatcontains 525 horizontal scan lines every {fraction (1/30)} of a second.This is termed a raster scan read out sequence. For cameras using theEuropean (PAL) format, the read out rate is 525 horizontal scan linesevery {fraction (1/25)} of a second. These figures translate into thefollowing standards: 1 frame is read out every 33 milliseconds forNTSC-format cameras, and 1 frame is read out every 40 milliseconds forPAL-format cameras. These standard read out rates hold true for a widevariety of video-rate cameras, whether vidicon tube cameras or solidstate cameras. The latter camera uses photodiodes for photodetection.For a raster-scanned video camera, the inter-frame time is the timebetween the raster scan of the last digitized scan line of one frame andthe raster scan of the first digitized scan line of the next frame.

[0091] One kind of solid state photodiode camera is a charge-coupleddevice (CCD) camera. The charge-coupled device of such cameras comprisesa large rectangular array of photodiodes deposited on a siliconsubstrate. Each photodiode is a sensor element separated and isolatedelectrically from its neighbors by a channel stop. During the period ofexposure, photons fall on the photodiodes while electrons in thedepletion layer of each photodiode move into an adjoining isolatedpotential well. Each well collects a number of electrons, and hencestores a specific charge. Once all the wells in the rectangular arrayare detected by the camera as being filled—thereby signalling the end ofexposure—read out of the stored charge occurs.

[0092] Read out for a CCD solid-state camera is done by shifting theelectrons across the wells in a systematic fashion to an output node,where the electron charge in each well is “read” in the same raster scansequence that the array of wells constituted in the photodiode array.The output node thus reads each well as having its own specific chargein a top-down, left-right sequence. The output node is connected to anamplifier that converts the charge in each well to a voltage, which isproportional to the stored charge in each well or pixel.

[0093] There are three most common arrangements of photodiodearchitecture in a charge-coupled device: full frame, frame transfer andinterline. Full frame CCD architecture means that the entire photodiodeframe is integrated, that is, exposed, to contain charge in all theframe wells before read out occurs. Consequently, for a full frame CCDcamera, the inter-frame time is the read out time of the chip.

[0094] Full frame architecture is currently used in slow-scan CCDcameras that employ a mechanical shutter. The shutter is opened duringimage acquisition and closed during charge transfer and readout. Theimplication of full frame architecture is that the image acquisitionrate is limited by the detector size of the photodiode array, the speedof the mechanical shutter and the read out rate of the image data fromthe charge-coupled device to the computer.

[0095] A camera using frame transfer CCD architecture divides the fullframe of photodiodes into two regions: an imaging section and a masked,storage section. During integration (exposure), the image is stored aselectron charge only in wells of the imaging section. Once exposure hascompleted, the stored charge is then shifted to the wells in the maskedsection very rapidly. After the stored charge has shifted to the maskedsection, two processes co-occur: the imaging section has been freed upto start exposure again while the masked section reads out the storedcharge from the first exposure. Consequently, the inter-frame time ofthis kind of CCD camera is the time it takes the stored charge to shiftfrom the imaging section to the masked section. Frame transfer CCDcameras can perform the dual tasks of exposing and reading out almostsimultaneously.

[0096] In a camera with interline CCD architecture, the full frame ofphotodiodes are arranged in alternating columns so that one columncontains imaging photodiodes and the next column contains masked ones.During integration, a column of imaging photodiodes transfers the chargein its wells to the neighboring column containing masked photodiodessection. As in the operation of a frame transfer camera, after thecharge has been transferred to the masked columns, the unmasked columnsof photodiodes can begin exposure again. Also similar to a frametransfer camera, Interline transfer cameras that contain overlappingmasked and unmasked diodes perform image acquisition similarly as aframe transfer camera. For an interline CCD camera operating insequential mode, the inter-frame time will be the same as for a frametransfer CCD, that is, the read out time of the photodiode array. If theinterline CCD camera is operating in overlapped mode, the inter-frametime will be the shift time for transferring the charge stored under themasked section.

[0097] Configuration of the System and Asynchronous Operation of theExternal Devices

[0098] When external devices 340, 336, 316 and/or 320 operate at a rateeither faster than or similar to the acquisition rate of camera 330,external devices 340, 336, 316 and/or 320 can operate asynchronouslywith camera 330, which in turn has implications for the organization ofthe system. When external devices 340, 336, 316 and/or 320 configuredinto the system allow asynchronous operation with camera 330, there isno need for direct connections between camera 330 and the externaldevices. Therefore, in a system of the present invention, externaldevices 340, 336, 316 and/or 320 are connected, for example, viaconnections 283 and 261 to the system computer 211. These devices arenot connected directly to camera 330. Asynchronous operation occurs byhaving a software method of the present invention signal devices 340,336, 316 and 320 to change parameter values in concert with theacquisition routine of camera 330.

[0099] Exposition of a Method of the Present Invention

[0100] At its most fundamental, the method of the present inventionoperates in the following way: The researcher configures the system toinclude camera 330 and the selected external devices 340, 336, 316and/or 320. The researcher inputs the number of Z-positions and/orwavelengths at which images will be acquired. By so inputting, aresearcher is in effect selecting the number of frames that camera 330will acquire in any one stack during an experiment or acquisition event.For example, inputting 20 Z-positions and 2 wavelengths translates intoa stack of 40 frames. The stack can be processed by appropriateapplication software, such as MetaMorph available from UniversalImaging, as a sequence that displays continuous change in the selectedparameters of Z-position and wavelength.

[0101] Upon the researcher's command to begin image acquisition, themethod directs camera 330 to begin exposing at the first selectedZ-position and first selected wavelength. The image is digitized, aprocess well known in the art, then read out to computer 211 and storedas a digitized image in a temporary memory location or buffer. Whencamera 330 begins reading out the exposed image, the method directsexternal devices 340, 336, 316 and/or 320 to change values. Recall thatif piezofocuser 340 and wavelength changers 316 and/or 320 areconfigured into the system, the method will direct wavelength changer316 or 320 to switch to all selected wavelengths at the currentZ-position before directing piezofocuser 340 to move objective lens 324to the next Z-position. The method continues to direct external devices340, 336, 316 and/or 320 to change values at the beginning of the readout for each image until these devices have moved through the entire setof selected values.

[0102] Critical to understanding the method of the present invention isthat camera 330 is given free rein, so to speak, to acquire images at ornear its acquisition rate while the method is directing the operation ofexternal devices 340, 336, 316 and/or 320 to coincide with the read outfrom camera 330. Critical to one embodiment of a method and system ofthe present invention is the limitation that the operation rate ofexternal devices 340, 336, 316 and/or 320 be the same as or faster thanthe inter-frame time of camera 330. For example, a researcher may employa camera with a fast inter-frame time, that is, a frame transfer cameraor an interline transfer camera operating in overlapping mode, whichrivals the operation rate of piezofocuser 340 and/or wavelength changer316 and/or 320 at about 1 to 2 milliseconds. In this embodiment, camera330 will actually be acquiring images at the same rate as these devices340, 336, 316 and 320 are changing Z-position or wavelength.

[0103] When the operation rate of external devices 340, 336, 316 and/or320 is slower than the inter-frame of camera 330, the method cannotdirect external devices 340, 336, 316 and 320 to change Z-positionand/or wavelength while and as camera 330 is reading out the currentimage to computer 211. For example, using a slow wavelength changer toswitch between selected wavelengths will result in images that willcontain comingled wavelengths. To reduce image corruption, in thisembodiment a researcher uses a method that directs camera 330 to foregoexposing images at every frame but to skip a selected number of frames,for example, 3, before exposing the image. In doing so, the method ofthis embodiment effectively gives external devices 340, 336, 316 and/or320 extra time to change parameters and to catch up with the camera.

[0104] Specific Apparatus and Application Software

[0105] Specific apparatus that may be configured into a system of thepresent invention so as to operate asynchronously using a method of thepresent invention include the following: the lines of MicroMax andPentaMax cameras available from Roper Scientific, Princeton Instrumentdivision as well as the line of Coolsnap FX cameras, available from thePhotometrics division of Roper Scientific. Also, the system and methodof the present invention may be operated with any camera complying withthe RS-170 or CCIR standards so long as it is interfaced to theprocessor through a digitizer card called a video frame grabber. Anexample of a video frame grabber that can create a software interrupt sothat the method of the present invention can direct a camera to acquireimages simultaneously as the external devices are varying is theFlashBus MV PRO PCI bus-mastering digitizer card.

[0106] Piezofocusers that may operate within the system and method ofthe present invention include the PIFOC® line of Microscope ObjectiveNanopositioners available from Physik Instrumente. Suitable wavelengthchangers that may operate within the system and method of the presentinvention include the following: Lambda DG5 Illuminator, Lambda DG4Illuminator, Lambda 10-2 Wavelength changer and Lambda 10 WavelengthChanger, all available from Sutter Instruments; Polychrome IIMonochromator available from TILL; Monochromator available from KineticImaging; and DeltaRAM Monochromator available from PTI.

[0107] Application software that would support asynchronous operation ofa camera and external devices include MetaMorph™ and MetaFluor®.

[0108] Exemplary Information Handling SubSystem

[0109] With reference to FIG. 1, when information handling subsystem 200comprises a computing device 211 in the form of a personal computer, anexemplary subsystem 200 of the present invention may employ a processorof the Intel Pentium III class, which currently includes 550 MHz, 700MHz, 750 MHz, 800 MHz or 850 MHZ. Preferred motherboards include eitherthe P3BF or CUBX av available from Asus. For display adapters, aresearcher may the ATI Expert@Play 98—8 megabyte; the Matrox G400—32megabyte dual display or the ATI Rage Fury Pro—23 megabyte.

[0110] Recommended memory requirements for the configured system are 256megabytes to 1 gigabyte SDRAM memory. Regarding the various drives:recommended is a WDC WD205BA IDE (20.4 gigabyte) hard drive availablefrom Western Digital; a 1.4 megabyte 3.5 floppy drive; and a CD-R58Sread/write 8×24 SCSI CD-ROM drive.

[0111] Recommended cards include—for the SCSI card: the Adaptec 2930Ultra SCSI2 kit; for the Input/Output card: the SIG Cyber PCI—1 serial,one parallel port; for the Network Card: the 3COM 3C905TX-M 10/100 PCI.

[0112] Method

[0113] With continuing reference to FIGS. 1 and 3, FIG. 5 shows a flowchart of a method of the present invention. Although a method of thepresent invention is described hereinbelow in terms of a computerprogram module for acquiring images at substantially the maximumacquisition rate of a camera while and as external devices to the cameracontinuously change image acquisition parameters, those skilled in theart will recognize that the invention may be implemented in combinationwith other program modules, routines, application programs, etc. thatperform other, particular tasks.

[0114] The image acquisition parameters that may be varied while and asan image is acquired include the Z-position, the excitation and/oremission wavelengths, and the vertical position of the microscope stage.The Z-position may be changed either by moving relative to the sampleobjective lens 324 using a piezofocuser 340 or by moving microscopestage 326 using stage mover 336. The image acquisition parameters asshown in FIG. 5 include the Z-position and the wavelength.

[0115] Configuring the System

[0116] Before camera 330 begins acquiring images, the researcher willalready have established the research goals and prepared specimen 332and will be ready to configure, that is, identify to the system, thespecific hardware the system will be using. As shown in FIG. 5, a methodof the present invention commences at step 500 with the researcherconfiguring a system of the present invention by selecting specificimage acquisition devices. Configuring the system is actually theselection of appropriate device drivers, which are software filespre-stored in an image acquisition/processing program that containinformation needed by the program module to operate the correspondinghardware. Configurable image acquisition devices include camera 330,means for changing the Z-position, which include piezofocuser 340 andstage mover 336, and wavelength changers 316 and 320.

[0117] With continuing reference to FIGS. 3 & 5, FIGS. 6A-D showexemplary user dialogs for configuring an automated optical microscopesubsystem 300. FIG. 6A shows an exemplary dialog for installing camera330. Shown to the left is a list of available camera drivers 610contained in an exemplary software package for imageacquisition/processing 245. Highlighting a particular driver 612 fromthe Available Drivers 610 list and clicking the Add 614 button resultsin the selected driver appearing in the list of Installed Drivers 616.The system as exemplified in the Driver Configuration 618 window isconfigured to contain only one camera.

[0118]FIG. 6B shows an exemplary dialog for configuring parametersrelating to the installed camera in FIG. 6A, particularly Exposure Time620 and Camera Shutter 622. Exposure Time 620 means the total exposureduration and is exemplified at 620 as 100 ms. The user may also selectat 626 to have the camera shutter remain open during the exposure time.By so selecting and by configuring a wavelength changer (316, 320) intosubsystem 300 as shown below in FIG. 6C, a researcher can operatewavelength changer (316, 320) as a shutter for system 300.

[0119] It is important to note that mechanical shutters external tocamera 330 such as shown at 308 or 314 in FIG. 3 must run at a cycletime greater than 25 ms because they are driven by a high voltage thattakes time to dissipate. Running these shutters at a cycle lengthshorter than 25 ms will cause a build-up of heat, leading to eventualjamming. For that reason, it is useful to allow a fast wavelengthchanger, with a 1.2 ms switch time, to operate as a shutter. This allowscamera 330 to acquire images at or near its maximum acquisition rate andthereby to promote asynchronous operation with external devices 340,336, 316 and/or 320.

[0120]FIG. 6C shows an exemplary dialog for installing an externaldevice into the system. Highlighting the name of a specific device at632 and clicking the Add 634 button results in device 632 appearing inInstalled Devices 636. As shown in FIG. 6B, two external devices havebeen installed in an exemplary system, a piezofocuser at 632 and awavelength changer at 638.

[0121]FIG. 6D shows an exemplary dialog for configuring a piezofocuserinstalled into system 300. On the bottom of the dialog are shown threebuttons 644, 646 and 648 by which a researcher determines the range ofthe distance over which piezofocuser 340 may move objective lens 324. Byclicking on Set Home 646, a user sets the Home position to whichpiezofocuser 340 moves objective lens 324 at the end of imageacquisition. Home is an arbitrary position and depends on the researchgoals. For example, a researcher may have determined by eye-focus orfrom processing previously acquired stacks events a certain criticalZ-position; such a Z-position would be set as Home. The Set Top 644position is the maximum upper limit above Home to which objective lens324 may move; the Set Bottom 648 position is the maximum lower limitbelow Home. Together the Top, Home and Bottom positions define the totaldistance over which piezofocuser 340 may move objective lens 324 duringthe exposure time. A researcher can initialize the starting position ofobjective lens 324 by keying in a Current Position 642. The MoveIncrement 640 option allows a user to select the incremental distancebetween Z-positions, which, as shown, is set at an exemplary 1.0micrometers.

[0122] By setting the Top and Bottom positions at 644 and 648 as well asthe Move Increment at 640, a researcher can calculate the total numberof Z-positions at which images will be acquired. For example, in FIG.6D, the total vertical distance over which a piezofocuser will move is22 micrometers; at an incremental distance of 1 micrometers, images willbe acquired at a minimum of 23 Z-positions (the start position=1 plus 22micrometers=23 total positions).

[0123] User Input

[0124] Referring back to FIG. 5, the next step in the method afterconfiguring the system is User Input at step 502. Here the user inputs arange of values so that the method can direct piezofocuser 340 andwavelength changers 316 and/or 320 to change to specific Z-positions andwavelengths during image acquisition.

[0125] Stream Acquisition Embodiment of User Input

[0126] FIGS. 7A-D show a first embodiment of user dialogs for inputtingZ-position and wavelength values, illustrated as the Stream Acquisition702 Embodiment. FIGS. 8A-H show a second embodiment of such userdialogs, illustrated as the Multi-Dimensional Acquisition 802Embodiment.

[0127]FIG. 7A shows that Stream Acquisition 702 embodiment contains fourseparate user dialogs or tabs—Acquire 704, Focus 706, Wavelength 708,and Camera Parameters 710—by which a researcher may input values foracquisition conditions. FIG. 7A shows the Acquire 704 dialog andillustrates a user's selections within that dialog. Options 718 and 720allow a researcher to check that camera 330 will be operating with apiezofocuser and a highspeed wavelength changer. By checking 718 and 720and by having already configured the specific devices into the system asshown in FIG. 6C, a user receives confirmation at field 730 that theinstalled devices support and are capable of asynchronous operation withthe camera during image acquisition.

[0128] In both the Stream Acquisition 702 and the Multi-DimensionalAcquisition 802 embodiments, a user may choose to acquire images whileand as both a piezofocuser 340 or stage mover 336 and wavelengthchangers 316 and/or 320 vary the Z-position and the excitation and/oremission wavelengths. This relates to the fundamental operatingprinciples of the method discussed hereinabove. Because external devices340, 336, 316 and 320 operate asynchronously with camera 330, theexternal devices do not wait for the camera to finish readout beforechanging their respective parameters. In one embodiment of the method,therefore, so long as each external device is operating faster or nearthe inter-frame time of the camera, several external devices may beconfigured together so that they are all operating to change their imageacquisition parameters as the camera is acquiring. In practice, however,although the camera and external devices operate asynchronously, theexternal devices operate synchronously relative to each other. Morespecifically, when, as shown at fields 718 and 720 in FIG. 7A, aresearcher selects to use both piezofocuser 340 and wavelength changer316 and/or 320, the method signals the piezofocuser 340 first to move toa new Z-position. The system of the present invention then signals thewavelength changer 316 and/or 320 to change to each selected wavelengthin successive frames. Only after a frame has been exposed at each of theselected wavelengths will the method signal to the piezofocuser 340 tochange Z-position. The sequential operation of the external devicesrelative to each other is more fully illustrated in FIG. 10.

[0129]FIG. 7A also shows that a user may select the number ofZ-positions to which piezofocuser 340 will move objective lens 324,illustrated at field 712 as 23. In theory, a user may select any numberof Z-positions at which to acquire images and is constrained by thebiological entity of interest and the research goals and not by thepresent method of operating a camera and external devicesasynchronously. The dialog in FIG. 7A works in concert with the dialogin FIG. 6D, which illustrates Top, Bottom and Home Z-positions.

[0130]FIG. 7A also shows an External Shutter 714 selection. Illustratedat 716 is an exemplary user's selection that a high-speed wavelengthchanger, the Sutter DG4, can serve as the external shutter for camera330. Recall that in FIG. 6B, a user may elect to keep the camera shutteropen during the entire exposure time. The External Shutter 714 selectionworks in concert with the dialog in FIG. 6B to direct a high-speedwavelength changer to function as an external shutter.

[0131]FIG. 7B shows the Focus 706 dialog in which a user may select thestarting and final Z-position of objective lens 324 as well as thedirection in which piezofocuser 340 moves during acquisition. The Focus706 dialog works in concert with FIG. 6D wherein the Top 644, Home 646,Bottom 648 and Current 642 positions are input. As illustrated in FIG.7B, at the start of image acquisition 732, piezofocuser 340 is at theTop 644 of the selected range, 646 (FIG. 6D). During image acquisition734, piezofocuser 340 moves objective lens 324 downward towards theBottom of the range, 648 (FIG. 6D). After image acquisition,piezofocuser 340 moves the lens 324 to the Current Position, 642 (FIG.6D), which for this example corresponds to Home as illustrated in 646(FIG. 6D).

[0132]FIG. 7B also shows the Plane Distance 738 to be an exemplary 1,which corresponds to the Move Increment 640 option in FIG. 6D. Further,the Total Distance 740, exemplified as 22, corresponds to the totaldistance calculated in the discussion above of FIG. 6D.

[0133]FIG. 7C shows the Wavelength 708 dialog for the Stream Acquisition702 embodiment of user input. As exemplified here, a researcher hasopted at 750 that the configured wavelength changer, as shown installedat 638 in FIG. 6C, will switch between 2 wavelengths during imageacquisition. The exemplary wavelengths include FITC at 752 and RHOD at754.

[0134]FIG. 7D shows the Camera Parameters 710 dialog. In selecting anAcquisition Mode 760, a researcher may choose between differentembodiments of a method of the present invention. For example, aresearcher may choose to direct camera 330 to acquire images at itsframe rate 762, that is, once the shutter is opened, exposure occurs atthe maximum operation rate of the camera. In an alternative embodiment,the researcher may opt to direct the camera to acquire images on anexternal trigger 764.

[0135] Further, in a different embodiment, a researcher may opt undercertain circumstances to direct camera 330 to expose an image only forcertain frames, done by inputting a Number of frames to skip, 766.Directing camera 330 to skip frames is useful when the researcher isusing an external device 340, 336, 316 and/or 320 that is slower thanthe read out rate of the configured camera. Because such externaldevices 340, 336, 316 and/or 320 are changing the Z-position andwavelength as camera 330 is exposing the image, the acquired stack willdisplay as garbled. An example of a system configuration when thisembodiment of the method would be useful is when a researcher is using afull frame CCD camera and a filter wheel, not a highspeed wavelengthchanger, to change wavelengths.

[0136] For example, by opting to have camera 330 skip 3 frames at 766, aresearcher is in effect directing the devices to change the Z-positionand/or the wavelength only on every fourth frame. If the rate ofexposure and read out for the camera used in the system were 25 ms andthe rate of operation of the selected wavelength changer were, forexample, 75 ms, opting to skip the devices changing parameters to everyfourth frame would give wavelength changers 316 and/or 320 the timenecessary to change the wavelength respectively.

[0137] MultiDimensional Acquisition Embodiment

[0138] With continuing reference to FIG. 5, at step 502 a researcher maychoose an alternative embodiment as shown in FIGS. 8A-G for inputtingvalues for various acquisition conditions. This is termed theMultiDimensional Acquisition 802 Embodiment. The distinguishing featurebetween the MultiDimensional Acquisition 802 embodiment and the StreamAcquisition 702 embodiment is that the MultiDimensional Acquisition 802embodiment allows the researcher to acquire sets of images of the samespecimen using the same acquisition conditions at regular timeintervals. The purpose of acquiring images under the same conditions atdifferent time intervals is to create a time lapse video.

[0139]FIG. 8A shows the Main 802 user dialog of the MultiDimensionalAcquisition 802 embodiment. The researcher must have checked each of theacquisition conditions—Timelapse 814, Multiple Wavelengths 816, Do ZSeries 816 and Stream 820—as shown in order for the correspondingdialogs of dialogs—Timelapse 806, Wavelengths 808, Z Series 810 andStream 812 to appear and be accessible to the researcher.

[0140]FIG. 8A also shows a Description box 822 in which a researcher mayinput a file title for the acquired set of images that will be storedidentified in permanent memory. As shown, the Description box 822illustrates that images in the exemplary experiment will be acquired atthree separate points in time, 10 seconds apart, as piezofocuser 340changes the Z-positions and as wavelength changer 316 and/or 320switches between 3 wavelengths. At Box 824, a user-identified name maybe input for the stored file that holds the acquired stack of images.

[0141] The Timelapse 806 dialog in FIG. 8B allows a researcher to inputthe “Number of time points” 830, here shown as 3, as well as the “TimeInterval” 832, here shown as ten seconds.

[0142]FIG. 8C shows that upon opening the Z Series 810 dialog, aresearcher can input all the pertinent information the program moduleneeds to direct piezofocuser 340 to change Z-positions during imageacquisition. This information includes the Current Position 840 ofpiezofocuser 340, the Increment 842 by which the program module willdirect the piezofocuser 340 to move to the next Z-position, the Top 846and Bottom 848 distances from the Current Position 840, the Step Size850 and total Number of Steps 852 the piezofocuser 340 will take duringimage acquisition. The Z dialog 810 allows a more direct inputting of ZSeries information than is done in the Stream Acquisition 702 embodimentas well as calculates directly the total Range 844 over whichpiezofocuser 340 moves objective lens 324 during image acquisition.Specifically, the MultiDimensional Acquisition 802 embodiment uses the810 dialog to input the same Z series information as is input in the twodialogs shown in FIG. 7B and FIG. 6D under the Stream Acquisition 702embodiment.

[0143] FIGS. 8D-F show the Wavelengths 808 dialog. A researcher canspecify in the # of Waves 860 box a maximum of 8 wavelengths that can beswitched between while camera 330 is acquiring images. As illustratedhere, 3 wavelengths have been selected. A researcher may type in aunique name for each numbered wavelength in the Name 864 box. TheCurrent Wavelength 862 box will indicate the list of input wavelengthnames and which wavelength the program module has directed wavelengthchanger 316 and/or 320 to start illuminating with when camera 330 startsexposing.

[0144] In FIG. 8D, the Current Wavelength 862 is listed as 1:DAPI; inFIG. 8E, as 2:FITC; and in FIG. 8F, as 3:RHOD. The names DAPI, FITC, andRHOD refer to abbreviations for fluorescent stains known in the art,each of which has a distinctive emission color when excited by a certainwavelength. Therefore, identifying a wavelength with the name of afluorescent stain that will become especially visible upon illuminationby that wavelength gives a researcher an easy mnemonic for identifyingvarious wavelengths. When inputting wavelength names, a researcher mustassociate an order with a specific name. That is, each input wavelengthis given a number that indicates the order in which it will illuminateduring acquisition. In this way, by keying in and assigning an order toeach wavelength, the program module can create a wavelength table.

[0145] When the user has configured a piezofocuser and a wavelengthchanger that operates fast enough so that the program module can directthese devices to change Z-position and wavelength as the camera isacquiring images, fields 866 and 868 appear checked as shown in FIGS.8D-F. In effect, the checks at 866 and 868 give notice and confirmationto the user that system 300 is configured with camera 330 and externaldevices 340, 336, 316 and/or 320 that can be used to perform the methodof the present invention.

[0146]FIG. 8G shows the Stream 812 dialog, which, to reiterate, appearsonly if the user has clicked on the Stream 820 option in the Main 804user dialog. Clicking on this option and configuring the system with anappropriate camera and external devices notifies the informationhandling system 200 that the method of the present invention may beinvoked, thus it serves as a further notice to the user. At heart,dialog 802 serves to give the researcher a single-page summary of thesystem parameters or dimensions that the method will execute after theresearcher has initiated image acquisition. Shown at field 870 is a listof dimensions that will be varied during image acquisition, which asillustrated are the Z-position and wavelength.

[0147] At box 872, the Exposure Time of the entire Acquisition durationis illustrated as 50 ms. The Stream Illumination 874 box allows theresearcher at 882 to select from a list of configured wavelengthchangers, which may be used to vary wavelength during acquisition. Asillustrated, a Sutter DG4 has been selected.

[0148] At 876, the researcher can select which memory location thedigitized images will be temporarily stored during acquisition,illustrated here as RAM.

[0149] Memory Allocation

[0150] With continuing reference to FIG. 5 and FIG. 7A, after aresearcher has input the required acquisition information, the programmodule determines at step 504 whether the information handling subsystem200 contains enough temporary memory to acquire and store the requestedstack(s) of images. In the Stream Acquisition 702 embodiment, theAcquire 704 dialog in FIG. 7A shows at 722 the memory requirements foran exemplary stack. As illustrated at field 728, the Total number offrames in the exemplary stack will be 46 and the amount of temporarymemory 722 needed to store the 46 frames will be 23.00 Megabytes. TheAmount of memory available 724 is exemplified as 255.50 Megabytes. Theresearcher is thereby notified that the system contains enough temporarymemory to acquire the exemplary stack of 46 images given the systemparameters as configured in FIGS. 7A and B.

[0151] With continuing reference to FIG. 8G, in the MultiDimensionalAcquisition 802 embodiment of user input, the researcher is informed atfield 878 of how much temporary memory the exemplary stack will demand,illustrated as 34.50 Megabytes, and at field 880 how much memory isavailable in the exemplary information handling system, illustrated as255.50 Megabytes.

[0152] Initialization

[0153] With continuing reference to FIGS. 1, 3 and 5, once the systemhas been configured, specific values input and memory allocationperformed, the program module at step 506 directs computer 211 todetermine if the Z-position will be varying during stream acquisition.If so, in step 508, the program module initializes the startingZ-position by moving piezofocuser 340 or stage mover 336 to the positionspecified by the user as the Start Position. For the Stream Acquisition702 embodiment, this is the Start At 732 field in FIG. 7B, which couldbe either the TOP 644 or BOTTOM 648 position as illustrated in FIG. 6D.For the MultiDimensional Acquisition 802 embodiment, refer hereinaboveto the discussion of elements 846 and 848 in FIG. 8C. At step 509, theprogram module creates a table of Z positions, discussed above in thedescription of FIG. 8D.

[0154] At step 510, the program module directs computer 211 to determineif the wavelength will be varying during stream acquisition and if so,to move the wavelength changer 316 and/or 320 to the position selectedby the user as position 1 in step 502. For the Stream Acquistion 702embodiment, this is the Wavelength #1 field, element 752 in FIG. 7C. Forthe Multi-Dimensional Acquisition 802 embodiment, refer above to thediscussion of element 862 in FIG. 8D.

[0155] In step 513, the program module creates a table of wavelengthvalues that specifies the order in which the wavelength changer 316and/or 320 will switch the wavelengths, which relates back to thediscussion of element 862 in FIGS. 8D-F.

[0156] In step 514, the program module initializes camera 330 by settingits frame number to zero and determines in step 516 whether camera 330has an external shutter 314. If so, the program module opens theexternal shutter in 518, and in step 520 directs the camera to wait acertain delay period to assure that the shutter has opened beforestarting to acquire images. The duration of such delay depends on theparticular shutter.

[0157] Image Acquisition

[0158] At step 522, the system is ready to begin acquiring images. Oncethe researcher initiates the acquisition routine, the camera starts toacquire images while and as the external devices operate asynchronouslyto vary the Z-position and/or the wavelength. For the Stream Acquisition702 embodiment, FIG. 7A shows the Acquire 732 button by which theacquisition routine is initiated. For the MultiDimensional Acquisition802 embodiment, the Acquire 826 button is shown in FIG. 8A.

[0159] The Acquire routine of step 522 is more fully depicted in FIGS. 9and 10 and described hereinbelow. A summary of step 522 is thefollowing: as an exposed frame is read out from camera 330 into atemporary memory location of computer 211, the program module of thepresent invention directs the Z-position and/or wavelength to incrementto the next position. The camera then exposes the next frame, therebycapturing an image that will show the changed Z-position and/orwavelength. During read out of that frame, the program module againdirects the incrementing of the Z-position and/or wavelength. After readout of the last frame, the program module ends acquisition of the stack.

[0160] After all the images have been read out and stored, in steps524-526 the program module directs external shutter 314—if employedduring acquisition—to close. In step 528, the stack of acquired imagesis copied out of the temporary memory location into a more permanentsystem memory location, thereby freeing system memory resources in 530to begin another acquisition event, if so desired, or ending the processat 532.

[0161] Different embodiments of the system of the present invention willstore the acquired images in different temporary memory locations. Inone embodiment employing a personal computer as the processor, atemporary memory location may comprise a buffer in the RAM of thecomputer. Other system embodiments may store the acquired images on areal time hard disk, which may include a Redundant Array of InexpensiveDrives (RAID), or other computer readable medium or, for embodimentsusing a distributed computing environment or for embodiments whereaccess to the program module is achieved via the Internet, on a local orremote server. Moreover, different embodiments of the present inventionmay store the acquired images in different permanent memory locations.Those skilled in the art will appreciate that different embodiments ofthe optical microscope system 100, especially as they relate to usingdifferent computer configurations 211 for processing the acquiredimages, may employ a variety of permanent memory locations for storingthe images.

[0162] Asynchronous Operation of Camera and External Devices DuringImage Acquisition

[0163] With continuing reference to FIGS. 2 and 3, FIGS. 9 and 10 showmore in detail the sequence of steps for acquiring images by camera 330operating at or near its maximum rate while and as external devices 340,336, 316 and/or 320 operate asychronously with the camera. At step 900in FIG. 9 the user initiates the Acquire routine of the program module,the whole of which corresponds to step 522 in FIG. 5. At step 902, theprogram module resets the frame number of the system to be 0. At step904, the program module begins the steps of image acquisition. At thispoint, the method of the present invention bifurcates into twointerrelated lines of processing, one line performed by camera 330 andrepresented by steps 912 to 916 and 922 to 928. The grey-filled shadingof the nodes at these steps indicate camera controller processing. Thesecond interrelated line relates to processing done within the maincomputer and comprises the steps of 918, 932, 936, 938 and 940. Steps932, 936, 938 and 940 are performed by the main processor and are soindicated by nodes with diagonal lines; step 918 is performed by thesoftware interrupt handler, which is so indicated by cross-hatching.

[0164] Image Acquisition and the Software Interrupt

[0165] At step 912, the program module informs camera 330 of the totalnumber of frames that the researcher has requested—equal to the numberof Z-positions multiplied by the number of wavelengths (See discussionof FIG. 7A)—and directs the camera to begin exposing the first frame.Simultaneously, computer 211 begins its line of processing at step 932,which is to determine whether the frame number has incremented.

[0166] Between the time that step 912 is finishing and before read outbegins at step 916, computer 211 is alerted that the exposure is endingby receiving a software interrupt—an instruction that halts computer 211from its continuous processing of the frame number. This alerting stepoccurs at node 914. At step 918, a software interrupt handler, which isa script in the Acquire routine of the program module, notifies computer211 to update the frame number, which is denoted by the broken line 919connecting node 918 and node 932. Once computer 211 increments the framenumber by 1, it returns to loop 933, where most of its processing timeis spent waiting for the frame number to increment.

[0167] At step 916, read out of the exposure image begins to a temporarymemory location, for example, RAM, via Direct Memory Access (DMA). Withreference to FIG. 1, DMA is the method whereby the camera interface card225 resident in computer 211 can transfer image data from camera 130directly into the memory of computer 211 without requiring theintervention of application software 245.

[0168] At this point, the program module can move through differentembodiments of the method depending on the configured camera. If theresearcher has configured a frame transfer CCD camera or an interlineCCD camera operating in overlapped mode into the system, then theprogram module progresses directly to step 926 while the exposed imageof the current frame is still being read out to temporary memory.

[0169] This embodied pathway depends on the special structure of thesecameras. As discussed hereinabove, the CCD device of a frame transfercamera has an architecture that allows a current frame to be acquiredwhile the previous frame is being transferred to temporary memory.Specifically, the frame transfer CCD device comprises both an imagingarea of photodiodes and a masked area, whereby the exposed image of theprevious frame is transferred very rapidly from the imaging photodiodearea to the masked area. The imaging area of the camera is thus freed tobegin exposing an image in the current frame before the previous imageis read out completely to temporary memory. In effect, the maskedphotodiode area represents a kind of stopgap storage by which the cameracan overlap the function of exposure with the function of read out. Theoverlapping of these two functions results in very rapid imageacquisition. An interline CCD camera operating in overlapping modefunctions similarly as a frame transfer camera and consequently has thesame rapid rate of image acquisition. The image acquisition rate ofvarious models of these kinds of digital cameras is not standardized aswith video cameras. For example, one frame transfer camera model listedin the Specific Apparatus section hereinabove, the PentaMax line,available from Princeton Instruments, has a frame transfer time of about1.5 ms.

[0170] Alternatively, if the acquiring camera is not a frame transferCCD or an interline CCD camera operating in overlapped mode, the programmodule moves through a different embodiment of the method. That is,before moving on to step 926, the program module executes step 924 bywaiting for the exposed image of the current frame to be completely readout. This pathway is employed when, for example, a full frame CCD cameraconstitutes the configured camera in the system.

[0171] The structure of the specific camera also has implications forhow the software interrupt is generated in step 914. The method of thepresent invention requires that the camera controller or the PCIinterface card 225 be capable of alerting computer 211 in the transitionperiod after the exposure has completed and read out is commencing. Inthe case of digital cameras, the camera controller so alerts computer211 by generating a software interrupt at the appropriate transitionmoment. Different camera manufacturers may term the transition momentdifferently. For example, Roper Scientific, which manufactures thePrinceton Instruments and Photometrics lines of scientific grade digitalcameras, terms this transition moment as Beginning of Frame or BOF, anddefines it as the start of read out.

[0172] In order for a software interrupt to be generated when videocameras conforming to the RS-170 or CCIR monochrome or RGBspecifications are configured into the system, a PCI digitizer card (225in FIG. 1) or frame grabber must be configured into the system and becapable of generating a signal in the form of a software interrupt whenthe frame grabber has completed digitization of the current frame. Atcompletion of digitization of a frame, an event known in the art as avertical blank, or v-blank, is generated. Integral Technologies,manufacturer of the FlashBus Mv-Pro frame grabber card, has implementeda software interrupt when the v-blank occurs. Other manufacturers offrame grabber cards may also generate a software interrupt for thisevent. The method of the present invention uses the software interrupt,generated either at the BOF of a digital camera or the v-blank of thevideo frame grabber, as the signal for alerting the computer 211 methodthat camera 330 has completed exposure of one frame.

[0173] After the program module has moved to step 926, the cameradetermines whether the last frame has been exposed. If so, the cameraends acquisition in step 928. If not, the camera continues and exposesthe next frame.

[0174] Changing the Z-Position and/or Wavelength and the SoftwareInterrupt

[0175] With continuing reference to FIG. 9, a critical point to grasp inthe method of the present invention is that step 916 and step 936 areco-occurring. That is, as the camera controller, in step 916, istransferring the exposed, digitized image to a memory buffer of computer211, the program module in step 936 is executing the routine shown inFIG. 10. Recall that in step 918 the software interrupt handler, ascript in the program module, causes the frame number to be incremented.Upon receiving a software interrupt from the camera controller or videoframe grabber in step 914, the software interrupt handler interrupts,through pathway 919, the processing subroutine 933 of system computer211 to notify the computer to increment the frame number by 1. With theupdating of the frame number at step. 932, the program module proceedsto step 936, which is the routine executed in FIG. 10.

[0176] To summarize, step 936 as shown in FIG. 9 comprises the routineshown in FIG. 10 for varying the Z-position and/or wavelength, which isexecuted by computer 211 at the same time that camera 330 is executingsteps 916 through 924 in FIG. 9. The co-occurrence of step 936 withsteps 916 through 924 constitutes the asynchronous operation of thecamera with the external devices.

[0177] In FIG. 10, at step 952 the program module queries if the systemwill be varying the wavelength during acquisition. If not, the programmodule moves to step 964 to determine whether the Z-position will bevarying during acquisition.

[0178] If the system has been configured to vary wavelength, eitherunder the Stream Acquisition 702 embodiment as illustrated in FIG. 7C orunder the Multi-Dimensional Acquisition 802 embodiment as illustrated inFIGS. 8D-F, at step 954 the program module queries whether theilluminating wavelength for the last frame was the last one listed inthe wavelength table. If not, in steps 956 and 960, the program moduledirects the wavelength changer 316 and/or 320 to change the wavelengthto the next one listed in the Wavelength Table. At this point, themethod goes back to step 938 in FIG. 9, at which point the programmodule determines whether to execute a new exposure at the changedwavelength or to end acquisition.

[0179] Alternatively, if the wavelength for the last frame were the lastone listed in the Wavelength Table, in steps 958 and 962, the programmodule directs the wavelength changer 316 and/or 320 to change thewavelength to the one listed at the start of the table. This means that,at this particular Z-position, each of the selected wavelengths hasilluminated the image in different frames.

[0180] At this point, the method progresses to step 964 where theprogram module queries whether the Z-position will be varied during theacquisition of this stack of images. If no, at step 968 the methodreverts back to step 938 in FIG. 9, wherein the program moduledetermines whether the last frame has been acquired.

[0181] If the Z-position is varying during this acquisition event, atstep 966 the program module determines whether to change the value ofthe Z-position. If objective lens 324 or stage 326 is already at thelast value listed in the Z-position table, in step 972 the programmodule resets the next Z-position to be the first value in the table.Alternatively, if the last value in the Z-position table has not beenreached, the program module in step 970 increments the value of theZ-position by 1. In step 974, the program module directs piezofocuser340 or stage mover 336 to move to the next Z-position. At step, 976 themethod then reverts back to step 938 in FIG. 9.

[0182] Here the program module determines whether this was the lastframe. If not, the program module increments the frame number by 1 atstep 932 and re-enters, at step 936, the routine of FIG. 10 to againchange the wavelength and/or the Z-position. The routine of FIG. 10 isperformed until the program module determines at step 938 that the lastframe has been exposed.

[0183] As discussed above in regards to the MultiDimensional 802embodiment of user input, a researcher may select to acquire a set ofimages at different points in time. If different time series areselected, as illustrated at field 830 in FIG. 8B as 3, the programmodule will calculate the number of frames as equal to the number ofZ-positions times the number of wavelengths times the number of timepoints. Thus, to continue with the above example, for 23 Z-points and 3wavelengths and 3 time points, the total number of frames equals 23×3×3,or 207 frames.

[0184] At step 940, the program module ends the Acquire routine and themethod progresses to step 524 in FIG. 5. As discussed hereinabove in thedescription of FIG. 5, the method progresses from steps 524 through 530,which comprise closing the external shutter, if any, copying the stackof images from temporary to permanent memory, thereby freeing temporarymemory and completing the method.

[0185] Post-Acquisition Processing

[0186] After all the images have been acquired and stored in permanentmemory, the method has been completed. However, after the completion ofthe method, a researcher can then process the acquired stack in avariety of known ways by suitable application software to createobservations that previously have not been possible to make. Forexample, a very important and valuable way of processing the acquiredstack is to play it back as an uninterrupted sequence of images, thatis, as a “movie”, that shows the variation in focus plane andilluminated light as continuous.

[0187]FIG. 11 shows an example of the Review MultiDimensional Data 1100user dialog in MetaMorph™, a software application that can process theacquired stack of images, by which a user can select to display theacquired stack of images as a continuous stream, that is, as a movie.Clicking on the Select Base File 1102 button in FIG. 11 allows aresearcher to select the file containing the desired stack of images tobe processed. Recall that at field 824 in FIG. 8A a researcher can inputusing the method of the present invention an identifying file name forthe acquisition event. Having selected a file, a researcher can requestin the Wavelengths 1104 box that the selected file of images display asbeing illuminated by certain wavelengths. As shown here in 1104, aresearcher may check any or all of those illuminating wavelengths thatwere selected in FIGS. 8D-F, illustrated as DAPI, FITC and RHOD. Eachchecked wavelength as illustrated in 1104 appear in its own window.

[0188] Z table 1106 is a two-dimensional array of all of the framesacquired at selected Z-positions in different time series. The number ofcolumns shown in Table 1106 equals the number of time series input bythe user. As illustrated at field 830 in FIG. 8B, the number of timeseries is 3, which corresponds to the number of columns in Table 1106.The number of rows in Table 1106 corresponds to the number ofZ-positions input at field 852 in FIG. 8C, exemplified there as 23.Thus, column 1 in Table 1106 represents the 23 Z-positions acquiredduring the first time series, column 2 represents the 23 Z-positionsacquired during the second time series, and so on.

[0189] To view an individual frame acquired at a certain Z-position in aparticular time series and illuminated by one of the checked wavelengthsin Box 1104, a researcher clicks on a particular cell of Table 1106. Theimages corresponding to that Z-position for that time series for eachwavelength checked in 1104 are displayed in Box 1122. As an example,highlighted cell 1120 corresponds to all the checked wavelengths at thefifth Z-position of the second time series.

[0190] To view a movie of all the Z-positions of a certain time series,a researcher highlights a cell in the 1106 array, say in column 1, andclicks on the appropriate arrow buttons at 1108 to play forwards andbackwards through the 23 Z-positions of the first time series. To viewthe images of a certain Z-position through through time, a researcherhighlights a certain cell, for example, cell 1120 at the fifthZ-position, and clicks on the appropriate arrow buttons at 1112 to playforwards and backwards through the 3 images of Z-position #5 in thethree time series.

[0191] Clicking on the Load Images 1114 button collates all the selectedframes as a subset of the originally-acquired stack. In this way, thesubset stack may be played back as a movie to view the change in thatparameter through time. Even more importantly, by clicking on the SelectBest Focus 1116 button, a researcher can initiate an autofocusingalgorithm for all Z-position images of a certain time series in order todetermine which Z-position, in other words, which focus plane, containsthe best-focused image. When the algorithm finds the best focusposition, an “X” will be placed at that location, as illustrated at1120. The autofocusing continues until a table of best focus positionsfor each time series has been created, illustrated by the “X's” at 1120,1124 and 1126. The researcher can then play these frames using theappropriate buttons at 1112 or click on 1114 to assemble these framesinto a subset stack, that can be played back as a movie of the bestfocus positions throughout time.

[0192] Although this discussion of post-acquisition processing of astack of frames acquired using the present invention does not describeclaimed elements of the present invention, it has been included toexplicate how the present invention provides a previously unknown kindof image set which researchers can process in known ways so as to createobservations of biological events at the cellular level that could nothave been made previously.

[0193] Although the invention has been particularly shown and describedwith reference to certain embodiments, those skilled in the art willunderstand that various changes in form and detail may be made withoutdeparting from the spirit and scope of the invention.

What is claimed is:
 1. A method for acquiring images using an automatedoptical microscope system, comprising the steps of: configuring anoptical microscope system which comprises a camera, a microscope, aninformation handling system and a device for altering an imageacquisition parameter; acquiring images at a rate substantially close tothe maximum image acquisition rate of said camera; and altering, duringimage acquisition, at least one image acquisition parameter whichapplies to the next image; wherein the configuring step comprisesinitializing a range of values over which said image acquisitionparameters will vary during the acquiring of images.
 2. The method ofclaim 1, where the at least one image acquisition parameter beingaltered is focus plane, light intensity, excitation wavelength oremission wavelength.
 3. The method of claim 2, wherein the at least oneimage acquisition parameter being altered during image acquisition isexcitation wavelength or emission wavelength, whereby a stack offluorescence images is acquired.
 4. The method of claim 1, wherein theconfiguring step further comprises initializing a duration of timeduring which images will be acquired.
 5. The method of claim 1, wherein,during acquisition of at least one image, more than one imageacquisition parameter which applies to the next image is altered.
 6. Themethod of claim 5, wherein, during acquisition of at least one image,excitation wavelength and emission wavelength which apply to the nextimage are altered.
 7. The method of claim 1, wherein the informationhandling system comprises a memory, further comprising the step ofstoring a stack of images in the memory.
 8. An automated opticalmicroscope system programmed to contain a computer program product thatexecutes the steps of claim 7, comprising: a microscope, a camera, aninformation handling system comprising a memory, and a device foraltering one or more of the image acquisition parameters of focus plane,excitation wavelength or emission wavelength.
 9. An automated opticalmicroscope system programmed to contain a computer program product thatexecutes the steps of claim 1, comprising: a microscope, a camera, aninformation handling system, and a device for altering one or more of ofthe image acquisition parameters of focus plane, excitation wavelengthor emission wavelength.
 10. The automated optical microscope system ofclaim 9, wherein the microscope comprises an objective lens and anobjective lens positioner, and wherein the computer program productcontains programming for directing the objective lens positioner toreposition the objective lens between images.
 11. The automated opticalmicroscope system of claim 9, wherein the microscope comprises anexamination site and an examination site positioner, and wherein thecomputer program product contains programming for directing theexamination site positioner to reposition the examination site betweenimages.
 12. The automated optical microscope system of claim 9, whereinthe microscope comprises a wavelength selector for selecting theexcitation wavelength or emission wavelength or both, and wherein thecomputer program product contains programming for directing thewavelength selector to re-select excitation wavelength or emissionwavelength or both between images.
 13. The automated optical microscopesystem of claim 12, wherein the wavelength selector is a monochromator.14. The automated optical microscope system of claim 12, wherein thewavelength selector is a filter wheel.
 15. The automated opticalmicroscope system of claim 9, wherein the microscope comprises a shutterand wherein the computer program product contains programming forcontrolling the shutter.
 16. An automated fluorescence imaging systemcomprising: a light source; a light source wavelength selector; aspecimen examination site; an optical system; an optical systempositioner for changing the position of at least a portion of theoptical system relative to the specimen examination site; a fluorescenceemission wavelength selector; a camera; means for acquiring images fromthe camera at a rate substantially close to the maximum imageacquisition rate of the camera; and a processor for automaticallycontrolling one or more of the light source wavelength selector, theoptical system positioner or the fluorescence emission wavelengthselector while a stack of images is being acquired.
 17. The automatedfluorescence imaging system of claim 16, wherein the optical systempositioner comprises means for adjusting the focus plane of the opticalsystem.
 18. An automated imaging system comprising: a specimenexamination site; an optical system; an optical system positioner forchanging the position of at least a portion of the optical systemrelative to the examination site; a wavelength selector; and means forautomatically controlling either or both of the wavelength selector orthe optical system positioner while acquiring a stack of images, thewavelength selection or optical position being changed between imagesand the images being acquired at a rate substantially close to themaximum image acquisition rate of the camera.
 19. The automated imagingsystem of claim 18, further comprising: a fluorescence emissionwavelength selector, and means for automatically controlling thefluorescence emission wavelength selector while acquiring a stack offluorescence images.
 20. The automated imaging system of claim 19,wherein the fluorescence emission wavelength selector is a filter wheel.21. The automated imaging system of claim 18, wherein the optical systempositioner comprises means for adjusting the focus plane of the opticalsystem.
 22. The automated imaging system of claim 18, wherein thewavelength selector is a filter wheel.
 23. The automated imaging systemof claim 18, wherein the wavelength selector is a monochromator.
 24. Theautomated imaging system of claim 18, further comprising a mechanicalshutter in the optical system.
 25. An automated method for acquiringimages comprising the steps of: providing an optical system whichcomprises optical elements, a camera and a means for changing an imageacquisition parameter; acquiring a stack of images at a ratesubstantially close to the maximum image acquisition rate of the camera;and changing an image parameter between images, the change beingtriggered by the beginning of the read out of an image.
 26. Theautomated method for acquiring images of claim 25, wherein an imageacquisition parameter that is changed is focus plane and the opticalsystem further comprises means for changing focus plane.
 27. Theautomated method for acquiring images of claim 25, wherein an imageacquisition parameter that is changed is excitation wavelength and theoptical system further comprises at least one of a monochromator or afilter wheel for changing excitation wavelength.
 28. The automatedmethod for acquiring images of claim 25, wherein an image acquisitionparameter that is changed is emission wavelength and the optical systemfurther comprises a filter wheel for changing emission wavelength. 29.The automated method for acquiring images of claim 25, wherein theimages in the stack are fluorescence images.
 30. The automated methodfor acquiring images of claim 29, and between at least one pair ofimages, the image acquisition parameters of excitation wavelength andemission wavelength are changed.